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Method for Stereoscopic Measuring Image Points and Device for Carrying 
Out Said Method 

The invent ion relates to stereometry, particularly to non-contact methods of object spatial 
characteristics determination by ids stereoscopic images and can be used ra pho! r mm .1 
medicine, construction, architecture, biology, systems of object identification, natural sources 
research, assessing risks of natural and man-caused disasters and the effects thereof and for other 
purposes. 

There is a known method of stereoscopic measuring image points, which consists of a 
stereoseopk m isurin tereo-model by determining the position of the aiming axis of eyes, 
relative to the main optical axis of the monitoring system and tracking the monitoring results by 
determining the position of the aiming axis of eyes, which is the closest to the one churned 
herein (SU No. 55 1504, G01C 1 1/04, 25,03.5977). 



The weak points of sascl method are: 

- Absence of the visual control of the stereo-images monitoring process, and, therefore, 
low measuring productivity. 

- Low precision of determining the position of the aiming axis of ey es at the moment of 
eye focusing. 

Eye fixation is determined by the physiological features of the human vision; it is a 
transitional process of a sight fixation at the time of focusing on any point of the subject during 
certain period of time. The said method implies the detection of the focus moments on the base 
of die amplitude analysis and of position speed change of the aiming axis of eyes, but because 
the human eye continuously makes micro-movements, sight fixation is not a geometric point 
which is stable in time and space, but a zone of undetected shape, which is being formed in some 
range of lime. Since the concrete points with the discrete data are required for the 
photc m iefn< 1 frucut ihe se of fixations den nmed 1 the >et >d described ioe 
allow detecting the position of the aiming axis of eyes with the required for the pho tog ram metric 
measuring precision. 

On the other band, because of the individual lateral particularities, the movements of the left 
and tight eye are not synchronic, that is why the selected fixations do not determine die moment, 
when both eyes look at the same point of the subject. 

The principal scheme of the device (SUNo. 551504, GO 1C 1 1/04. 25/03/1977), which allows 
carrying out the method of stereoscopic measuring the linage points, is already known. 

The described principal scheme of the device consists of the photogrammetric device of the 
analytic type, containing the processor inside; with the built-in system of image entry into the 
televisioi > 1 ' ■ \ the b se of vidicons 

The device is assigned to measure die photographic images using the prism-cube with partly 
silver-plated internal edge for the construction of the observer's eye image in the television 
automate vidicon ocular. 
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The weak points of the principal scheme of the described device are: 

- lack of the "feedback" system (i.e. reflection of the measuring data on the images 
themselves), which leads to lack of observation control. 

lack of precise compensation mechanism of the head movement, because the 
compensation offered in the head movement device is based on the measuring of an eye 
glare, which is formed by infrared sources of radiation and does not take into 

o the eye which causes nonlmo j c 
change of the glare position at the time of eye movement; besides, while the head 
position is changing at the time of focusing on the point of the image, the eye makes 
some compensating movements, which also leads to the nonlinear modification of the 
glare position: 

- impossibility of the precise discrete fixation of the measuring data with the use of the 
systems, which are built on the base of the television automates with the analog vidicons; 

- low precision of the detection of the eye center pupil, which is caused by low contrast of 
the human eve images, if the Ymht sources assij ted onl fot naw > hi tenting are 
used; 

- losses of the optical radiation at the time oi pa atn in igh the prism-cube with the 
partially silver-plated internal edge: 

- small range of vision of the optical systems, which are used in analog photogrammetric 
devices. 

There is a device of the precise detection of the sight direction (A Precise Eye-Gaze 
Detection and Tracking System, A. Perez, M.L.Cordoba, A. Garcia, R Mendez, MX. 
Munoz, JL. Pedra/a, F. Sanchez, W SCO '2003. February 3-7, 2003 Pken, Czech Republic), 
carrying out the method of Eye-Gaze Detection, which is the closest to claimed herein. 

The described device consists of the surveillance camera for tracking eye movements, 
panoramic camera for tracking head movements, system for image entry into personal 
computer, and four infrared radiators for forming special glare-marks on the eye surface. 

The weak points of the described device are lack of precise compensation of the head 
movements and necessity of use the video-cameras with a very high definition. Movements 
of the head are detected m the device by the images of observer's face (in particular, by 
identification of the eyes on the entire face image), which is done by the wide-angle video- 
camera for tracking the head. Pails of the face have some relative movements, that is why 
they can not be used as the stable basic points of the head, and that is why the method used in 
the device is not Strict and, therefore, cannot comply with the requirements for high-precision 
measuring. The device presumes the use of only one video-camera to receive the eye images, 
but because the eyes of observer are located at some distance from each other, in addition to 
the images of the eyes there is insignificant information data, capturing by the camera (image 
of the part of the face in the bridge of nose area). Therefore, in order to receive precise 
enough images, it is necessary to increase the requirements for the camera's definition 
capacity, i.e. to use matrix of large dimension in the video-cameras, but increase of the 
matrix dimension leads to increase of the volume of the incoming video-information and 
noticeably increasing the requirements for the productivity and speedy action of the video- 
capture plate. "The invention solves the problem of increasing the productivity of measuring 
the spatial characteristic of the object by its images on the stereoscopic pictures. 

The problem can be m>1\u > s Mowing: accordn e invention in the method of 
stereoscopic measurin in sge points, including construction of a stereoscopic model based 
on two overlapping images, detection of the position of the aiming axis of eyes in 
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stereoscopic perception of that model and recording the observation results at eye fixation 
moments, the projection of the sight fixation area on the monitor screen with the images 
observed is computed and the typical points of the observed object, corresponding with those 
areas, on the fragments of digital stereo-.niiar.tes, are selected 



("here are add) I c i >ut1 ! I i hi is soi able 

- to identify typical points of the same name of the observed subject, which arc selected on 
the fragments of the digital stereo-images, correlating with areas of sight fixation, for the 
left and right eye by time synchronization; 

- To identify typical points of the same name of the observed subject, which are selected 
on the fragments of the digital stereo-images, correlating with areas of the sight fixation, 
for the left and right eye. starting with the condition of crossing of the corresponding 
rays, determined by the vectors coplanarity equation; 

- to do the calibration of the system before starting the observation, by observation of the 
image with test-objects with the known position data m the system of the position data of 
the main monitor, comparing the position data of the pupils of the eyes, determined in the 
system of position data of the video-camera, with the position data of the test objects, 
shown on the main monitor, and the subsequent mathematic dependencies, describing 
mutual transformations of position data. 

- At the time of system calibration, to position the test objects for observation in different, 
conditions (for example, time, duration and order of appearance of the test objects, 
disposition, size, shape and color of the test objects, surrounding background, static or 
dynamic conditions of the test object appearance); 

- while observing, to visually control the measuring data on the screen of the main monitor 
by imprinting the color markers into the image area, coordinating with that fixation; 

- To do a visual, control of measuring on the main monitor screen by modifying the color 
parameters of the area on the stereo-image, corresponding with that fixation; 

- To do the compensation of the observer's head movements by computing the movement 
in the position of the aiming axis of the eyes by images of certa in parts of the observer's 
head. 

- To do the compensation of the head movement of the observer by track ing several marks, 
fixed on the head of the observer; 

- To track the head movement by the marks fixed close to eyes in a way to get the images 
of those marks captured by the video-cameras, recording the observer's eyes movements: 

- To make the marks for tracking the observer's eye movements in a special (for example, 
ellipsoid) shape, what allows detecting precisely the position and orientation of the mark, 
and, accordingly, movements of the observer's head. 

- To detect the position data of the motion of the head in two mutually perpendicular 
planes; 

To detect the pupil of the eye position while recording the eyes movements in three- 
dimensional space by receiving two images of each eye by two synchronized video- 
cameras, fixed with different foreshortening. 

The problem can be solved by die additional input, according to the invention, the 
construction made in the shape of an eyeglasses frame with the specially shaped marks, 
positioned on vertical plane, into the content of the device for stereoscopic measuring image 
points, consisting of two video-cameras for recording the eye movements, video-cameras for 
tracking the head movements, system of the image video-capture by the personal computer, 
monitor for displaying the Image, system of stereo-surveillance, allowing to observe 
stereoscopic images, in a way to get captured by the eye movements recording cameras 
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Additional versions of the device are possible, wherein it is reasonable to do the 
following: 

- To install the additional specifically shaped marks, located on the horizontal plane, into 
the eyeglasses frame and to install a mirror, placed above the observer s head, into the 
device; in which connection to install the video-c tmei . foi trad ig the hi id m >• s icnts 
in a special way for capturing at the same time the part of the head and the reflection in 
the eyeglasses frame mirror with she specially shaped marks placed on the horizontal 
plane on it ; 

- To install in the system, in addition to the main two video- cameras for tracking the 
movements of each eye separately, two additional video-cameras, placed in a way to 
synchronic^ iy record the movements of each eye by the mam and the additional video- 
cameras from two points. 

- To install an additional monitor for visual tracking the observation and operation of the 
observation process; 

- To install the system of the infrared highlighting of the eyes area; 

3 t! i ec i ft] ( 1 t x tmera c > e pat te 1 htt in tl 
visible range of spectrum; 

The mentioned above advantages and also the particularities of that invention explained 
by the best versions of it's carrying-out with the references to the enclosed figures: 

Figure No J - typical trajectories of the eye at the time of sight fixation while focusing on the 

point of the object; 

Figure No, 2 scheme of the stereoscopic observation of the stereo-Images on the monitor 
screen; 

Fig. No. 3 and Fig. No. 4 - general view of the device for measuring the three-dimensional, 
position data based on its stereo-images; 

Fig. No. 5 - eyeglasses frame with the specifically shaped marks (for example, ellipsoidal 
shape), for the recording of the head movements; 

Fig. No. 6 scheme of locations of the main and additional video-cameras for the recording 
of the eye movement in three-dimensional space. 

Construction of t he three * e objec in the real time * 

on it's visual copy on two Hat stereo-images can be done by tracking the observer's eyes 
micro-movements and recording the sight fixations with the subsequent computing the 
multitude of the points of intersection of the com <] I - ed) rays, determinating the 
homogenic virtual surface which is identical to the geometric surface of the object. 

The determination of the sight fixations can he done by dividing the bask consecution of 
the eye movement contents into areas of fast movement (saccades) and areas of the sight 
stabilization (fixations) separately for each eye. On the fig. I The typical trajectories of the 
eye are pictured at the time of the sight stabilization while focusing on the point of the object 
-areas of fixations 1 (marked with dotted lines). As a rule, detection of the point of fixation 
2 ( highlighted by the firm line) can be done by computing the simple geometric average or 
the average weighted centroid of the points of the sight trajectory- in the limits of the fixation 
area I (fig.!), however, as it is shown on the fig J. for that, the problem of vagueness of 
choice of the concrete points of the fixation 2 occurs, which is caused by significant 
dispersion of the points of trajectory of the eye movement m the limns of fixation area 1 . 

It is suggested to solve that vagueness in the following way. Because the purpose of the 
stereoscopic measuring is to determi . >h ni d characteristics of the object, the observer 

>a he iypk ect. making t je t i < < nvironmei 

and determining it's shape and size. It is naturally to assume, that the projection of the sight 
fixation area on the ob erved image on the monitoi c.i en eontaii s 'fie oi several those 
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points. Position data of those points on the digital image can be found automatically, 
applying the Harris algorithms, KLT or similar. 

in piacticc. the al rithms desc >ed can select severa tyj cal joints on the image 
fragment, corresponding with that fixation area. Because a hui a s >h i logically can 
not stabilize his her sight on two different points of object, it is suggested to synchronize by 
time the fixation points, selected in the fixation areas of the left and right eye with the use of 
algorithms for searching the typical points on the image. Time synchronization allows to 
decrease ambiguity of the detection of the typical points on the image in the limits of the 
fixation area, corresponding to the focusing on the object by two eyes at the same time, 
however, because of the lateral particularities of the certain individual (dynamic asymmetry 
of one of the eyes, i.e. some "delay", "gap", which is similar to the "right-hander, left- 
hander" effect}, that direction still can not comply with the actual state of the corresponding 
rays, pre-existing at the time of focusing on the concrete point of the object. 

That ambiguity is solved by the analysis of the geometric intersection of the aiming rays 
of the left and right eye at the time of stereoscopic focusing on the point of the object. 

The characteristics of the human binocular vision arc such, that horizontal spatial parallax 
P between the pair of the corresponding points as and a r on two images, located on the same 
plane ( with the condition of their separate observation b\ the eves og J) causes tn human 
being the sensa tion of perception of the certain point, located out of the plane. As it is shown 
on the fig.2, the plane D is a display plane with the stereo-images, on which the observer's 
eyes are focusing, and axis Ri and R y are the vision Axis which are corresponding with the 
left L and right R eye, and B is an eyes base. While focusing separately on two 
corresponding points ai and a f , reflected on the left and right image of the stereo-pair, the 
image of the point A of the object's virtual model, formed as a result of intersection, of the 
sight Axis Rj and R r . is formed in the human brain. The condition of the geometric 
intersection of the corresponding rays is determined by the condition of belonging of the 
vectors Rj and R, to the same plane, passing through the eyes base B. That condition is 
written by the vectors copiaoarity equation: 8 (Rj x R r ) - 0 

Therefore, in the method suggested, the points-'candidates" 2. selected in the limits of 
the corresponding fixation areas for the left and right eyes, first have to be synchronized, and 
then have to be cheeked for compliance with the condition of vectors eomplanarity. The 
belongings of the corresponding vectors to the same primary plane is a strict geometric 
condition for observation of the certain point on the stereo-image, and, therefore, the 
multitude of the points of intersection of the corresponding (conjugate) rays, satisfying the 
condition of the eomplanarity, while focusing on the stereomodel, determines the homogenic 
virtual surfa ad object 

To cany-out the suggested method of stereoscopic measuring image points, the device 
(fig. 3 and fig.4) is offered, containing video-cameras 3 and 4 with infrared color filters 5 and 
6 for recording the eyes movements 7 and 8 accordingly, the video-camera 9 and the mirror 
10 to track the head movements 11, system of video-capture the linage by the personal 
computer, the Main Monitor 12 for output of the stereo- image under review, the additional 
(controlling) monitor 1 3 to visually control the process of observation and to operate the 
observation process, system of stereo-surveillance 14, system of infrared eyes highlighting 
15 and eyeglasses frame 16 with the specially shaped, marks, for example, in ellipsoidal 
shape, Hand 18 (fig. 5) 
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Fig. 5 represents the eyeglasses frame 16 with the spec 5c .or example 

in ellipsoidal shape. 17 and 18, the marks 17 are located in the vertical plane in a way to 
make their images to get captured by the corresponding cameras 3 and 4, recording the eyes 
n sots ? and t vrdiogi i the spe 1 > i , i 

in a way to get their image captured through the mirror 10 by the video-camera 9. The 
images 17 and 18 are used for the head movements tracking. 

The scheme of the location of the main 3 and 4. and the additional video-cameras 19 and 
20 for tracking the eves movements 7 and 8 in the three-dimensional space is represented on 
the fig 6. 

The device for measuring the spatial characteristic of the object by its stereo-images 
works the following way. 

For observation of the object based on its stereo-images, the stereoscopic images are 
displayed on the screen of the main monitor 12 and the calibration of the system has to be 
done for the certain observer, at the meantime the observer observes static and dynamic test 
objects, displayed on that monitor. The main idea of the calibration is to determine the 
dependencies between the position data of the pupils of the eyes centers, captured by the 
video-cameras 3 and 4 at the moments of sight stabilization during the observation of the test 
objects on the monitor screen 12, and the position data of those objects with the subsequent 
consideration of the psycho-physical particularities of the certain observer at the time of 
observation and analysis of those results. The calibration can be done either in monocular 
regime (both eyes focusing on the mono-image of the test, objects on the monitor screen), or 
in stereoscopic regime (focusing on the virtual models of the three-dimensional test objects, 
using the system for stereo-view). 

Directly; the observations has to be done by focusing on the stereoscopic images of the 
observed object with the fixation of the sight trajectory with the consideration of the 
calibration results, detection of the fixation areas and points with the control by the 
condition of coplanarity and the following determination of the spatial position data of the 
object. The determination of the spatial position data of the points of the object surface can 
be done by the analysis of the lengthwise Parallax P with the use of the set of the two- 
dimensional position data of the corresponding points in the fixation areas on the base of 
transformations, which are used in photogrammetry or projective stereometry. The 
construction of a three-dimensional model is done by orientation of the virtual mode! 
constructed, relative to the set of the fixed basic points, assigning the external system of the 
position data of the object. 

The compensation of the head movements is reaiizaed b deti tn tation of the 
movements factors by the computed movements of the image with the use of the special 
marks 17 and 18 and entry of the corresponding compensating amendments in the position 
data of the pupil of the eye, the camera, tracking the head movements, must be synchronized 
with the video-cameras, tm • - 1 e e s movements. 

The use of the additional video-cameras 19 and 20 for tracking the eyes micro- 
movements allows to determine the three-dimensional position of the pupil of the eye and to 
increase the precision of the sight direction computing. 

t ol >e ol t - i _j i e feedbael nc vhen the fixa 

areas with the correctly calculated location of the point ol intersection of the corresponding 
rays are marked on the screen of the controlling monitor 1.3 by imprinting the color 
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markers, and on the screen of the main Monitor 12 by changing the color parameters of the 
part of the image, corresponding with that fixation rhe sible fot the 

observer not only to control the progress of work (i.e. to see the areas of the image, in which 
the review is already done), but to estimate the quality of the e s m as well analyzing 
the color of th mark imprinted into the image on the conti " ttonito] 1 3 (die color of 
the markers is determined by the values of divergence of the residual vertical parallaxes, 
calculated with the condition of coplanarity and corresponding with the certain points of 
fixations ) Because the mecb mi m oi feet back diows the areas whose the observations have 
been already done, the control results also can be used at the time of recommencement of 
work after interruption. 

The claimed method and the device of stereoscopic measuring image points can be 
utilized industrially in computer systems, assigned for digital stereoscopic measuring as well 
as in the fields like digital interaction photometry, image detection, three-dimensional 
measuring in medicine, biology, natural sources research, mine workings, natural sources 
workings, assessing risks of die natural and man-caused disasters and the effects thereof, 
interactive teaching systems, systems for stereo-vision tests, system of professional aptitude 
tests, computer and television games. The industrial adaptability of the invention has been 
proved by the tests of the sample of the device, carrying out the claimed method. 
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